Papers with deeper mechanistic understanding of
Where Confabulation Lives: Latent Feature Discovery in LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Despite advances in large language models, hallucination remains a critical failure mode . despite these advances, LLMs are prone to hallucinic outputs that contain illusory information presented as if it were factual or grounded in reality. |
| Approach: | They propose to isolate and analyze confabulation, a foundational aspect of hallucination, where the model fabricates facts about unknown entities. |
| Outcome: | The proposed method reveals that the model can fabricate facts with minimal disruption, shedding light on the inner representations that drive factual and non-factual output. |